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Abstract 


At Turkish universities, schools of foreign languages provide compulsory and voluntary 
language education for college students before they start their programs. This study aims 
to evaluate the program implemented for the English preparatory classes at Istanbul 
Medeniyet University, School of Foreign Languages during 2016-2017 academic year. This 
study has a mixed approach. 131 students and four instructors evaluated the program in 
terms of aspects such as the course materials, course contents, testing methods, 
assignments and the instructors. The data were collected via a questionnaire and semi- 
structured interview questions. The analysis of the questionnaire data was conducted via 
descriptive statistics and a descriptive analysis was carried out with the data from the 
interviews. The findings revealed that the participant students were indecisive about the 
effectiveness of the language program in general. Moreover, it was seen that the mean 
scores of the female students, the intermediate level students and the ones attending the 
program voluntarily were higher when compared with the males, the elementary level 
students and the ones attending the program obligatorily. 
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istanbul Medeniyet Universitesi ingilizce hazirlik programimin 


degerlendirilmesi 


0z 

Tirkiye’deki iiniversitelerde, hazirlik okullari O6grencilere lisans egitimine 
baslamadan 6nce zorunlu ve istege bagli dil egitimini vermektedir. Bu calisma 
istanbul Medeniyet Universitesi Yabanci Diller Yiiksekokulu’nda 2016-2017 egitim 
dZretim yilinda verilen ingilizce hazirlik egitimini degerlendirmeyi amaclamaktadir. 
Calisma karma desendedir. 131 6grenci ve 4 6gretim gorevlisi verilen egitimi ders 
materyalleri, ders icerikleri, 6leme y6ntemleri, Gdevler ve 6gretim gérevlileri gibi 
acilardan degerlendirmistir. Veri bir anket ve yari yapilandirilmis gorusme sorulari 
araciligiyla toplanmistir. Anketin analizi betimsel istatistiklerle yapilmistir ve 
gortisme verileri betimsel analiz ile incelenmistir. Bulgular géstermektedir ki 
dgrenciler genel manada programin etkinligi konusunda kararsizdir. Ayrica, kadin 
Ogrencilerin, orta seviye (intermediate level) Ogrencilerin ve istege bagli olarak 
programa katilan 6grencilerin memnuniyetinin erkeklere, baslangic seviyesi 
(elementary level) 6grencilere ve zorunlu olarak programa katilan d6grencilere gore 
daha yiiksek oldugu gériilmistir. 
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Introduction 

Curriculum design and development not only in language teaching but also in other 
subject areas are regarded as critically important elements of education. Therefore, much 
research has been conducted in this field for different reasons, such as offering suggestions for 
practitioners to present a common base or to understand the effectiveness of the implemented 
curriculum. Quicke (1999, p.1) proposes that “a curriculum provides a framework for learning.” 
Since curriculum suggests what we have to teach by giving reference to the educational needs 
of the students, it includes the “how” and the “what” of learning, which takes place both 
formally and informally inside educational institutions. 

The curriculum process is not effortless or straightforward because of its nature. 
According to Demirel (2004, p. 17-18), foreign language learning, which is a cumulative 
process, combines cognitive behaviors and new psychomotor skills. Students are expected to 
acquire a foreign language system by developing these cognitive behaviors and motor skills 
through a well-organized foreign language curriculum. However, it is not uncommon to 
observe mismatch among different components of the suggested curriculum like objectives, 
content, and teaching and learning process. Nunan (1988, p. 138-141), for instance, emphasizes 
the frequent mismatch between what the curriculum offers and what occurs in the language 
classroom. Tollefson (1991, p. 99-100) explains the inconsistency between theories and 
classroom practices through the ‘modernization theory.’ He claims that even though some 
educationists widely criticized traditional instruction and many classes shifted to ‘modern’ 
methods in which students are supposed to take more responsibility for their learning, many 
teaching practices still place students in paradoxical circumstances. 

From a slightly different perspective, Brown (2009) makes recommendations based on 
his thirty-five years of experience in language curriculum development. He suggests that any 
language needs analysis must include both language and contextual needs and therefore 
program-based assessment must be directly connected to the students’ needs, the course 
objectives, the materials being used, and the activities that are going on in the classrooms 
(Brown, 2009, p. 87). Besides, he shares his experience in Turkey when he was invited to 
evaluate a new English curriculum prepared for 11 million students throughout the country. 
He did not only find materials covered in the curriculum confusing but also cultural content 
like the royal family, cricket, Benjamin Britten problematic. He adds, “that is when I began to 
realize that the materials development team had not done any needs analysis or consulted with 
any real live teachers and students” (Brown, 2009, p. 95). This first-hand experience proves the 
effectiveness of his curriculum development process theory, which starts with the 
identification of situation, context, students, or analyzing needs. In his model, the steps of 
formulating goals, determining course content, designing course units and modules, 
constructing lesson plans follow the very first step as well as teaching, assessment, and revision 
of curriculum steps as final ones (Brown, 2007, p. 151). Indeed, it is possible to study ‘the 
curriculum’ from several different perspectives as offered by Nunan (2002, p. 4). Accordingly, 
curriculum planning can be examined at a decision-making level concerning different aspects 
of it, starting from identifying learners’ needs and purposes. Another perspective can be in 
‘action’ which takes the evaluators into the classroom itself. Apart from these, a perspective 
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relates to assessment and evaluation gives the program evaluators a chance to make 
recommendations for respondents to improve things in the future. 

Consequently, if we summarize the role of evaluation with Rea-Dickins and Germanie 
(2003, p. 20), “evaluation makes teachers aware of parameters in which they are working. 
Raising awareness in this way helps them to analyse the context for possible openings for 
innovation or for constraints”. We expect that the study conducted at Istanbul Medeniyet 
University, School of Foreign Languages, does not only help to raise awareness for the 
curriculum evaluation process within this specific institution but also makes contributions to 
other foreign language schools. 

This paper aims to answer the following questions in order to evaluate the program 
implemented for the English preparatory classes at Istanbul Medeniyet University, School of 
Foreign Languages during the 2016-2017 academic year: 


1. How efficiently do the students at Istanbul Medeniyet University (IMU) School of Foreign 
Languages find the English preparatory program implemented during 2016-2017 academic year 
in terms of the courses, the course materials, assignments, the testing and evaluation processes, 
the academic staff, perceived learner engagement, other aspects of the instruction and the 
system? 

2. Do the students’ opinions about the English preparatory program vary significantly in terms 
of their gender, preparatory program status (compulsory or voluntary), language level? 

3. What do the instructors think about the English preparatory program implemented during 
the 2016-2017 academic year considering the course books, curriculum, physical and technical 
conditions, and students’ performance? 


Literature Review 

The main aim of this paper is to evaluate the implemented program of English as a 
Foreign Language at a state university. For this purpose, we will report related literature and 
studies focusing on teaching English as a foreign language. We can say that inherent in the 
term of curriculum evaluation is the concept of perspective. Thus, it is clear that a considerable 
amount of research has been conducted in this field in light of different perspectives to make 
an accurate evaluation, and they have proposed some suggestions despite its complexity. 
Sullivan (2006, p. 591) states that “when we evaluate, we are saying that the process will yield 
information regarding worthiness, appropriateness, goodness, or validity”. Moreover, because 
the curriculum implementation process requires a severe evaluation, including opinions 
gathered from different groups such as students, teachers, material developers, it causes the 
content of related studies to be widespread. For instance, the study conducted by Coskun 
(2013) aimed to evaluate the modular intensive general English language teaching program 
applied at a university in Turkey. In his research, he collected the qualitative data through 
interviews with 22 instructors working in the preparatory school, and the quantitative data 
from 381 preparatory school students through a questionnaire adapted from Tung’s (2010) 
study. The research question of “What are students’ and instructors’ perceptions of the 
effectiveness of the English preparatory program concerning three program dimensions: course 
materials, the teaching, learning process, and the assessment?” was tried to be answered. The 
results showed that the materials prepared for different skills and sub-skills were positively 
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evaluated, whereas many students expressed concerns about the materials for speaking. In 
addition to this, some instructors claimed that grammar, reading, and writing materials do not 
help students for their further education. It was also found that instructors mainly used 
lecturing instead of role-plays, discussions, and presentations. As the study tried to investigate 
the effectiveness of a modular system, the instructors emphasized the low motivation level of 
the students and discipline problems. 

Similarly, Kocyigit and Erdem (2018) undertook a study to review English preparation 
classes in higher education considering the graduate research. Their study found that these 
studies usually employed a quantitative research design and they generally focused on 
students’ perspective. Accordingly, there is a gap in qualitative studies and there is also a lack of 
studies on instructors’ views. The current study satisfies both gaps by having a mixed-method 
design and focusing on instructors’ views as well. Furthermore, some instructors suggested 
more academic English in the program. Similary, Tekin (2015) adopted Tung’s (2010) data 
collection instrument for evaluation of prep school to evaluate the English Language Teaching 
and English Language and Literature preparatory classes of a state university in Turkey by 
interpreting triangulated data with qualitative and quantitative means gathered from 106 
students and two lecturers. The study revealed that even though the majority of participants 
were satisfied with the present language program, there is still a need for change. For example, 
students stated that the program provides opportunities to improve their grammar, writing, and 
reading skills but not speaking and listening skills and vocabulary knowledge. Another major 
issue raised by the participants was the selection of materials, especially for speaking and 
listening courses and the need for a separate vocabulary course. The findings related to 
teaching methods in the study revealed that traditional teaching methods such as “lecturing” 
and “question and answer” dominated the lessons. Therefore, communicative activities such as 
role-plays, students’ presentations did not find many places in the lessons (Tekin, 2015, p. 730- 
732). An extensive evaluation held both quantitatively and qualitatively by Unal et. al. (2017, p. 
616-617) to evaluate another public university’s English Language program reported that the 
current curriculum was successful, and the school was equipped with sufficient materials and 
facilities. However, both the instructors and students who participated in the study made some 
suggestions for a more effective curriculum. Technology integration into the classroom and 
teaching practice, revision of assessment, teaming instructors as material developers, testers, 
fewer class hours, and more English for Specific Purposes (ESP) courses instead of general 
English courses were the most suggested items. Lee (2016), likewise, found that the students 
felt more interested in ESP instruction, and they considered ESP courses could be more useful 
for their future needs in a study which investigated the perspectives and perceptions of 
students studying at a technological university in Taiwan. 

From a different viewpoint, aiming a kind of internal validity, Cetinavci and Zehir- 
Topkaya (2012) evaluated two different regimes of grammar teaching, one that followed its 
language content and another that followed the language content of the main course in a year- 
long preparatory English class at a Turkish state university. They interviewed the 
administrator, five instructors, and 36 students one-on-one and as groups. Also, they observed 
six classrooms and compared relevant exam results and attendance records. The analysis made 
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it clear that the program following the language content of the main course was favored much 
more strongly as a positive step in the course of producing the desired learning outcomes. 

Even though there is a high demand for preparatory school evaluation, a deep analysis 
of foreign language teaching and learning policy has to be invested. Thus, this necessity has 
taken the attention of some researchers in the field like Din¢er et. al. (2010), who explored 
viewpoints of university preparatory school students on the English Language Education 
process in Turkey. They aimed to advise teachers by taking into consideration students’ 
readiness levels and their experiences during their prior education periods. For this aim, 38 
preparatory class students from Tourism Vocational College Preparatory Program were asked 
to write a composition in order to learn their high school years’ language experience. Analyzed 
compositions of the students clearly showed that some frequently experienced problems that 
they transferred to the undergraduate level negatively affected their foreign language learning 
experience. The component named “foreign language teachers’ teaching capabilities,” including 
sub-components of being unaware of the current methods in language education, use of old- 
fashioned methods, their disregard towards the course was the most mentioned one. 
Curriculum and course materials component followed the teacher factor in students’ writings. 
The results showed that students were unpleasant with curriculum, English hours at the high 
school, and course books. However, students commented that they did not participate in the 
class effectively, revise what they were taught, and did not give much importance to English 
(Dincer et al., 2010, p. 239-241). 


Methodology 

In this study, the English module at Istanbul Medeniyet University School of Foreign 
Languages preparatory classes was examined from the following aspects: courses, course 
materials, assignments, testing and evaluation processes, academic staff, learner engagement, 
and some other aspects of the instruction and the system. The study followed a mixed method 
consisting of an interview and a questionnaire as data collection tools. A mixed methods 
research refers to the design that “focuses on collecting and analyzing both quantitative and 
qualitative data in a single study” (Hesse-Biber, 2010, p. 26; Creswell, 2009, p. 204). This study 
also adopts the concurrent triangulation approach which aims to reveal any convergence, 
differences or combination by comparing the quantitative and qualitative data (Creswell, 2009, 
p. 213-214). For the quantitative data of this study, the students were given a questionnaire 
that included items for the evaluation of the aspects mentioned above. For the qualitative part, 
four instructors were interviewed via a semi-structured interview form that covered questions 
about the same aspects of the language program. 


Context 

This study aims to evaluate the program implemented for the English preparatory 
classes at Istanbul Medeniyet University, School of Foreign Languages during the 2016-2017 
academic year. The program lasted 32 weeks in total, and this was divided into four periods 
each of which was 7 or 8 weeks. The students were given a placement test before the classes 
started and placed to an A level (elementary) or B level (pre-intermediate) class accordingly. 
The classes were reorganized after each period considering the grades of the students. That is, 
the students were grouped according to their language levels as well as their achievement in 
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the tests. The program did not follow a modular system where learners moved to the next level, 
like from Al to A2 or from A2 to Bl. They were instructed within the same level group 
throughout the year, and the terms of A and B referred to the groups rather than the language 
levels at the end of the year. In the last period, when the data for this study were collected, 
there were 9 language classes at school, and 7 were for A level students while only 2 were for B 
level ones. The English preparatory year was compulsory for the students admitted to programs 
in which the medium of instruction was English. The students who volunteered were also 
accepted to the preparatory school, although they would be entirely instructed in Turkish in 
their future programs. However, these two groups were instructed in separate classes. The 
students from the compulsory language program would be expelled from the school if they 
failed, whereas the ones from the voluntary language program could start their department 
even if they failed in the language class. 


Table 1. The courses and the instructional materials 


Compulsory Voluntary 
Courses Classes Classes Instructional Materials 
(Hours) (Hours) 

Main Course 16 18 New Language Leader by Pearson Education 

is Ready to Write by Pearson Education 
ate i ne Writing Pack 
Pre-faculty 2 2 Course Pack from Reading Passages 
Listening and 4 7 Real Listening and Speaking by Cambridge University 
Speaking Press 
Bending Fi . Real Reading and Reading Explorer by Cambridge 

University Press 

Terminology 2 x Materials created by the departments 
Total 32 24-26 


The program involves several components that will be briefly described below. First, 
the courses covered in the language program and the instructional materials for each course are 
shown in Table 1. As can be seen in Table 1, the language instruction was delivered in six 
different courses for the students from the compulsory program. They received general English 
instruction in Main Course classes, and they were instructed separately for reading and writing 
skills, whereas listening and speaking were presented in an integrated way in the same course. 
They also followed courses like Pre-faculty and Terminology in which they were provided 
English for Specific Purposes (ESP). The language instructors gave the Pre-faculty while for 
Terminology courses, academics from their departments were invited to make the students get 
familiar with the terminology they would need in their future programs. Listening and 
Speaking, Reading and Terminology courses were not provided for the students from the 
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voluntary program. The students followed the materials published by private companies like 
Cambridge and Pearson as well as some packs prepared by the Material and Curriculum Office 
in the school. The compulsory groups received 32 hours of instruction per week, and this 
decreased to 24-26 hours for the voluntary groups due to the different syllabus implemented 
for them. When it comes to the testing and evaluation processes, Table 2 presents a brief 
summary. 


Table 2. The testing and evaluation processes 


Evaluation process in each period Periods Proficiency Test 
1s period: 15% 
Quiz: 20% 2"4 period: 20% 
Mid-term: 40% 3"4 period: 25% 
Pop quiz: 5% 4% period: 40% B2 Level 
Assignments: 20% ae 
Writing portfolio: 15% J 


60+ 


As can be understood from Table 2, the students were evaluated based on quizzes 
(20%), a mid-term exam (40%), pop quizzes (5%), assignments (20%), and a writing portfolio 
(15%) for each period. However, the contribution of the average grade for each period to the 
final grade was not equal. Considering the difficulty of the instruction throughout the year, the 
students raised their grades from the Ist to the 4th period (15% for the 1st period, 20% for the 
2nd period, 25% for the 3rd period and 40% for the 4th period). The average grade calculated 
according to these ratios should be 60 or above so that the students were allowed to take the B2 
level proficiency test given at the end of the year. The students were required to take 70 or 
above to be considered as successful in the proficiency test. 


Sampling 

For the quantitative part of the study, a total population sampling technique was 
adapted. For this technique, the entire population is included in the research, so it is more 
commonly used when the number is relatively small (Etikan, Musa & Alkassim, 2016, p. 3). 
Therefore, the questionnaire was delivered to all the 154 preparatory class students studying at 
[MU in the academic year of 2016-2017. Out of these 154 students, 131 were included in the 
analysis, which makes up 85% of the population, since some were absent on the data collection 
date and some others did not fill in the questionnaire properly. The detailed information about 
the participant students is presented in Table 3. 


Table 3. The participant students 


Variables Groups N 
” iis] A 103 
anguage leve . 28 
: Compulsory 76 

Language preparation status 
Voluntary 55 
Gender Female 80 
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Male 51 


Total 131 


It is understood from Table 3 that out of 131 participants, 103 were from A level, and 
28 were from B level classes. 76 of them received the preparatory language program since it 
was compulsory, and 55 of them volunteered for it since they were willing to learn English 
before they started their departments. Finally, 80 of these students were females, whereas 51 of 
them were males. 

For the qualitative part, purposive sampling technique was implemented. In this 
technique, participants are chosen for a specific purpose that is relevant to the research needs 
(Cohen, Manion & Morrison, 2005, p. 103). Based on this technique, only 4 of the 20 English 
instructors were included in the interview. Some of the instructors had been dismissed and the 
school had recently employed some of them. The four participant instructors were picked for 
the interviews since they were the ones who had experienced teaching the program under 
study. 

An official document was taken from the institution to get their permission for the data 
collection. The questionnaire was implemented when the classes were available during the 4th 
period. As for the interviews, an appointment was made with each participant instructor when 
they were available and they were visited in their offices. 


Data collection instruments 

In order to get the quantitative data, the participant students were asked to fill in the 
Preparatory Language Education Program Evaluation Questionnaire developed by the 
researchers. The items in the questionnaire were written based on the related literature, the 
program evaluation tools used by different schools of foreign languages in Turkey and the main 
features of the implemented program. The draft of the questionnaire was examined by different 
experts (two faculty members from ELT and Educational Sciences departments and one English 
instructor) and the items were revised according to their feedback. The final questionnaire 
consisted of 52 items with which the students expressed their agreement on a 5-point Likert 
scale: 1: Strongly disagree, 2: Disagree, 3: Neither agree nor disagree, 4: Agree and 5: Strongly 
agree. The items included statements about courses, course materials, assignments, testing and 
evaluation processes, academic staff, learner engagement, and some other aspects of the 
instruction and the system. The Cronbach’s alpha coefficient was calculated as .907 for the 
questionnaire and this value indicates a high level of reliability for the tool since a coefficient 
between 0.65 and 0.8 is considered acceptable in social sciences (Singh, 2017; Vaske, Beaman & 
Sponarski, 2016). 

As for the qualitative data, a semi-structured interview form was used by the 
researchers to interview the instructors. The form included seven questions that referred to the 
same aspects of the language program. The questions were directed to the sample group, and 
the biodata was also obtained. The interviews were audio-recorded and transcribed for the 
analysis. In order to increase the validity and reliability of the data collection tools, and thus 
this study, the data collection tools were prepared by first checking the preparatory class 
evaluation procedures of several other state universities. Then, the questionnaire and interview 
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questions were prepared by the three authors together, and several meetings were held to 
consider potential problems and ensure consistency. Finally, the interview questions were sent 
to an expert specialized in testing. The items of the questionnaire and the interview questions 
were adapted in accordance with the suggestions. 


Data analysis procedures 

For the analysis of the quantitative part, the data gathered via the questionnaire were 
transferred to SPSS 22 software, and Independent Samples T-test and Mann Whitney U test 
were conducted in addition to the calculations for means, frequencies, percentages, and 
standard deviation. The results were considered significant at the level of p<.05. 

As for the qualitative part, the descriptive analysis method was used to analyse the 
interview data. In descriptive analysis, the data are interpreted according to the pre-defined 
themes or may be grouped and reported according to the research questions. Direct quotations 
from the respondents are frequently utilized while reporting the findings. The point is to 
interpret and present the data in an organized way (Yildirim & Simsek, 2008). The data were 
categorized according to the interview questions and sub-themes were assigned. Then, as a 
content, the interview data in this study were analyzed using descriptive analysis in order to 
understand the preparatory English teaching in IMU through the four aspects: the course 
books, curriculum, physical and technical conditions, and students’ performance. 


Results 
In this section, the findings from the analysis are presented following the research 
questions. First, the findings from the quantitative analysis that reflect the students’ opinions 
and then the instructors’ views from the descriptive analysis are given. 


The questionnaire 

The first research question was stated as “How efficient do the students at [MU School 
of Foreign Languages find the English preparatory program implemented during 2016-2017 
academic year in terms of the courses, the course materials, assignments, the testing and 
evaluation processes, the academic staff, learner engagement, other aspects of the instruction 
and the system?”. 

The participant students’ opinions about the overall efficiency of the preparatory 
language education program are presented in Table 4. 


Table 4. Students’ mean satisfaction levels for the preparatory language education program 


Questionnaire n x sd 


preparatory language education program 


; : : 131 3.27 4931 
evaluation questionnaire 


Table 4 shows that the mean for the Preparatory Language Education Program 
Evaluation Questionnaire is 3.27 out of 5. That is, the students were reasonably pleased with 
the English Preparatory Program implemented during the 2016-17 academic year. They did not 
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consider the program as inefficient in general, but they also believed that it needed to be 
improved. Means for the courses covered in the program are given in Table 5. 


Table 5. Means for the courses in the program 


Courses n x sd 

Main Course 130 3.45 1.1687 
Writing 130 4.18 9627 
Pre-faculty 131 2.67 1.2355 
Listening and Speaking 122 2.59 1.2574 
Reading 118 2.64 1.2977 
Terminology 114 2.39 1.3333 


As can be seen in Table 5, the students found Main Course classes quite efficient 
(x=3.45) whereas they were not satisfied with Pre-faculty (x=2.67), Terminology (x=2.39), 
Listening & Speaking (x=2.59), and Reading classes (x=2.64). Writing was considered as the 
most effective and beneficial course in the program (x=4.18). Means for the course materials 
used for the language instruction are given in Table 6. 


Table 6. Means for the course materials 


Materials n x sd 

Language Leader (Main Course) 131 2.90 .7958 
Ready to Write & Writing Pack 130 3.38 1.2721 
Real Listening and Speaking 123 2.30 1.1157 
Real Reading & Reading Explorer 123 2.47 1.2369 
Supplementary Materials and Packs 131 3.07 .9866 


According to Table 6, the students were pleased mostly with the materials of writing 
classes (x=3.38) and the other supplementary materials and packs (x=3.07) that were prepared 
by the instructors of the related office. However, the mean scores show that they were not 
satisfied with the commercial materials chosen for Main Course (x=2.90), Listening and 
Speaking (x=2.30) and Reading (x=2.47) courses. Therefore, it is possible to conclude that the 
students were more pleased with the packs and supplementary materials prepared by the 
Material and Curriculum Development Office compared with the commercial products. Means 
for the assignments given to the students to support their language learning are presented in 
Table 7. 
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Table 7. Means for the assignments 


Assignments n x sd 

Writing assignments 131 3.09 9152 
Online assignments 129 2.31 1.3083 
Assignments in general 131 3.01 .8000 


Table 7 shows that the students were satisfied with the assignments they did for their 
writing course (x=3.09), but they did not find the online assignments efficient (x=2.31) for their 
learning. It is also seen that they were content with the assignments they were required to do 
throughout the year (x=3.01). Means for the testing and evaluation processes are presented in 
Table 8. 


Table 8. Means for testing and evaluation processes 


Testing and evaluation processes n x sd 

Use of English 129 3.53 .9846 
Speaking 131 3.67 1.0323 
Listening 129 3.13 1.1345 
Writing 131 3.83 1.0138 
Evaluation 131 3.24 8707 
Testing and evaluation in general 131 3.35 6792 


The students were tested on different aspects of language throughout the year, and 
Table 8 shows that they found the testing and evaluation processes efficient in general (x=3.35). 
It is also evident that they were satisfied in each component of the tests (use of English=3.53, 
listening=3.13, evaluation=3.24), but the highest scores were for the testing of productive skills 
(speaking=3.67, writing=3.83). Taking the items under this category into account, it is possible 
to conclude that the students considered the testing and evaluation processes appropriate for 
their level and coherent with the curriculum and also thought that the evaluation procedures 
were just. Means for the other aspects of language instruction at the School of Foreign 
Languages are shown in Table 9. 


Table 9. Means for the other aspects of language instruction 


Different aspects of language instruction n x sd 
Use of ITC in class 130 3.48 1.0942 
Pair and group work activities 130 3.49 1.1959 
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Re-grouping students for every period 117 3.55 1.3610 
Putting theoretical info into practice 131 3.55 8740 


Contribution of prep program to academic and professional life 131 3.81 9733 


When the means presented in Table 9 are considered, it is possible to argue that the 
students were satisfied with the integration of ITC in language teaching (x=3.48). They found 
the pair and group work activities conducted in the lessons efficient and helpful for their 
language learning (x=3.49). It is also clear that they were pleased with the way they were re- 
grouped according to their test scores for each period (x=3.55). Finally, it is understood that 
they believed they could put the theoretical info into practice (x=3.55) and this one-year 
language program would contribute to their academic and professional life (x=3.81). Finally, 
means for learner engagement and the academic staff are presented in Table 10. 


Table 10. Means for learner engagement and the academic staff 


Learner engagement and academic staff n x sd 


Learner engagement 131 3.10 1.0506 


Academic staff 131 3.76 7611 


As can be seen in Table 10, the students were content with both their effort and the 

instructors’ contribution to the program throughout the year. However, it is seen that they 
were more pleased with the instructors (x=3.76) than their performance (x=3.10). 
The second research question was formulated as “Do the students’ opinions about the English 
preparatory program vary significantly in terms of their gender, preparatory program status 
(compulsory or voluntary), language level?”. The t-test results that show a significant difference 
between the students’ opinions in terms of their gender are presented in Table 11. 


Table 11. The t-test results for the students’ opinions in terms of their gender 


t test 
Variables Groups n x sd 
t df P 
Pp l i Female 80 3.34 484 
reparatory language education 201 129 046" 

program Male 51 3.16 492 

Female 80 4.35 747 
Writing 2.28 128 .025* 

Male 50 3.92 1.192 
*p<.05 


Table 11 shows that the students’ opinions about the preparatory language education 
program in general (p=.046) and writing classes in particular (p=.025) differ statistically 
significantly in terms of their gender. When the means for the groups are checked, it is 
understood that the female students were more satisfied with both aspects (PLEP=3.34 and 
writing=4.35) than the males (PLEP=3.16 and writing=3.92). The t-test results that show 
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significant difference between the students’ opinions in terms of their preparatory program 
status are presented in Table 12. 


Table 12. The t-test results for the students’ opinions in terms of their preparatory program 


status 
t test 
Variables Groups n x sd 
t df P 
Compulsory 75 2.66 1.338 
Terminology 3.33 112 .001* 
Voluntary 39 1.85 1.163 
*p<.05 


As can be seen in Table 12, there is statistically significant difference between the 
students’ opinion about the Terminology course in terms of their preparatory program status 
(p=.001). When the means are compared, it is seen that the learners from the voluntary 
program found this course a lot more inefficient than the ones from the compulsory program. 
This finding can be explained by the fact that the voluntary students were not offered this 
Terminology course as well as the possibility that they thought they would not need any 
terminology in English since the medium of instruction was Turkish in their department. 
Finally, the Mann Whitney-U test results that show a significant difference between the 
students’ opinions in terms of their language levels are presented in Table 13. 


Table 13. The Mann Whitney-U test results for the students’ opinions in terms of their 
language levels 


. Language Mean of Sum of 
Variable fe. 8 N Ranks Ranks U Z P 
B Level 28 83.95 2350.50 
Ficpe sory Janet ik tavel 103 61.12 6295.50 939.50 -2.822 005* 
education program 
Total 131 
B Level 28 77.45 2168.50 
Main course A Level 102 62.22 6346.50 1093.50 -1.969 .049* 
Total 130 
B Level 24 72.21 1733.00 
Reading A Level 94 56.26 5288.00 823.00 -2.096 .036* 
Total 118 
B Level 24 70.88 1701.00 
Terminology A Level 90 53.93 4854.00 759.00 -2.302 .021* 
Total 114 
B Level 28 89.05 2493.50 
Language leader A Level 103 59.73 6152.50 796.50 -3.638 .000* 
Total 131 
; B Level 25 85.88 2147.00 
Ree esaine A Level 98 55.91 5479.00 628.00 -3.887 000* 
Reading explorer 
Total 123 
Supplementary B Level 28 84.25 2359.00 , 
materials A Level 103 61.04 6287.00 eas eal ag 
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Total 131 
B Level 28 82.84 2319.50 
Online assignments A Level 101 60.05 6065.50 914.50 -2.948 .003* 
Total 129 
B Level 28 79.36 2222.00 
Tests for use of English A Level 101 61.02 6163.00 1012.00 -2.433 .015* 
Total 129 
B Level 28 80.52 2254.50 
Listening tests A Level 101 60.71 6130.50 979.50 -2.567 .010* 
Total 129 
. B Level 28 81.48 2281.50 
aan WR gee 102 61.11 6233.50 980.50 -2.629 000" 
Total 130 
; B Level 28 78.98 2211.50 
Reeupeesndene!: cial 103 62.47 6434.50 1078.50 -2.087 037* 
for every period 
Total 131 


*p<.05 

As can be seen in Table 13, there is statistically significant difference between the A 
level and B level students’ opinions about the preparatory language education program 
(p=.005), Main Course (p=.049), Reading course (p=.036), Terminology course (p=.021), the 
instructional materials used for the Main Couse classes (p=.000), Reading classes (p=.000), the 
other supplementary materials (p=.004), online assignments (p=.003), the tests for Use of 
English (p=.015) and Listening (p=.010), pair and group work activities (p=.000) and re- 
grouping students for every period (p=.037). When the means are considered, it is seen that the 
B level students were more satisfied with the mentioned aspects than the A level learners. 


Results of the interviews 

This section summarizes the findings obtained through Descriptive Analysis from the 
interviews. In the first question, the aim is to learn about the instructors’ overall idea about the 
program. The participants think that the program is generally fine. They think that the 
strengths of the program are as follows: the lecturers are passionate and hardworking, and the 
program offers a variety of teaching which is useful for the students. As P3 states, “There is 
more variety of lessons. When I ask my colleagues in other universities, I think that we are one 
step ahead of them”. So, the instructors, in general, think that the program is better than other 
public universities. The participants also mention some of the problematic parts. They say that 
the program is not settled yet as it is a new university. Another problem mentioned is that the 
program is usually based on the book, not a needs analysis. Students also have a problem with 
motivation. As P3 explains, “J believe that students are really demotivated. In the beginning 
they come very motivated but after a few weeks they start to lose their motivation’. 

The second question of the interview is on the books used in the program (See Table 1 
for the books used). The analysis demonstrates that the English instructors are all happy with 
the books in a general sense. However, some issues are commonly mentioned by the 
interviewees. One significant finding is that they find the New Language Leader to be a 
successful book. However, they think that it causes some problems in their program mainly 
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because most of the students start with an Al level. New Language Leader, on the other hand, 
is focused on academic skills, and in this sense, it is sometimes too hard for the students. As P1 
argues, “The level of New Language Leader is a little bit demanding for our students as they 
come as true beginners.” As for the book selection process, the analysis shows that they are 
asked to mention their views, but their ideas are not taken into consideration most of the time, 
which is possibly due to the problems in formalities or practical issues such as lack of a 
sufficient number of instructors. For instance, P4 says “The way the process was applied was 
good but I guess there were formal problems. The books we suggested were not chosen. The 
process was good but the outcomes were not applied.” 

The next focus of the interview is the curriculum. The analysis suggests that the 
instructors are often happy with the curriculum. However, two main issues are mentioned. The 
first one is that the curriculum is overloaded and in turn pacing is fast. P3 explains that this is 
because “We have zero beginners.” The participants suggest that as some of the students have a 
really weak command of English, too many things are included in the curriculum. This may be 
a rational choice however, it puts too much stress on the students. P3 argues “They (the true 
beginners) are doomed to fail as the program is one year.”. She suggests that true beginners 
should study more than one year as it is usually not enough for attaining proficiency at B2 
level. The other problem mentioned is that the curriculum does not have obvious objectives 
and the focus of the books mainly leads it. As P2 argues “We only follow the book and each 
week we do a chapter or one and a half chapter. We have no goals.” So, it seems that following 
the structure of a book replaces an independent curriculum plan. Regarding this problem, P2 
thinks that the main reason is lack of instructors and a regular curriculum office. As the 
university suffers from lack of English instructors, the instructors in the curriculum office also 
teach a lot and accordingly, they cannot focus on curriculum development. 

The next issue in the interviews is the exams. The responses of the participants show 
that they find the exams useful and to the point. Pl expresses this as follows: “Regarding 
assessments the exams aim to test everything covered in the lessons and also we take sub-skills 
such as skimming, scanning and listening for details into consideration’. P1 goes on by saying 
“The frequency may be a little bit too many as far as I hear from the students. However, I think 
frequent exams force students to work harder, which is useful.”. This is a problem mentioned 
by the other participants as well. There are mid-terms, quizzes and pop-quizzes. The other 
instructors tend to think that this is a little bit too much but P1, as seen in her quotation, thinks 
personally that this is useful. The others say the number should be decreased as students are 
under constant stress due to having exams all the time. 

Another point raised in the interviews is homework. The analysis shows that all of the 
instructors are unhappy with the online homework system. Online system fails due to several 
issues such as internet connection problems in the university, students’ difficulty in accessing 
the internet and software problems in the online system of the books. P2 underlines a severe 
problem saying “For instance, a student responds correctly but the system says wrong. Why? 
Just because s/he did not leave space (between words) or s/he forgot a point or a question mark. 
It evaluates these kinds of issues as errors.’. The instructors were also asked about the 
contribution of homework. They think that the homework in the program is in generally 
helpful. One issue mentioned is that as the hour of teaching is very high in the program, 
students cannot find enough time to do the homework. P2 explains this problem as “ Weekly 
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hours should be less, because students this year have 22-28 hours of teaching and they cannot 
find time to do the homework.”. One final issue about the homework is whether sufficient 
feedback is given. This is related to the previous issue. The reason is as there is too much 
homework and as there are a lot of lessons, instructors also do not have enough time to give 
feedback. However, students prefer to get corrective feedback for their work and they even 
“feel cheated” when they cannot get feedback from their teachers (Harmandaoglu Baz, 
Balcikanh & Cephe, 2016, p. 63). The instructors acknowledge that they can give satisfactory 
feedback only occasionally. To exemplify, P4 says “Feedback is not given most of the time. For 
example, I gave them a worksheet and they only know their mark. We provide the answer key. 
... We were all in the office and we had to do some duties. The test office or curriculum office 
thinks whether to give feedback or do the duties in the office... Consequently, feedback is one 
step behind.”. However, P4 also suggests that feedback works well in only essay writing as it is 
taught as a process. 

Another issue is the physical conditions of the university and the classrooms. The 
answers demonstrate that the classrooms have some equipment such as projectors and 
smartboards and they are in general fine. However, the biggest problems are the frequent 
relocations between campuses and the physical conditions of the campus. P4 summarizes this 
as follows: “There was no canteen for a really long time. The students could not buy water or 
tea.” 

The final issue discussed in the interviews is students’ performance. The responses 
demonstrate that the students usually suffered from lack of motivation. Most of the participants 
agree that this is worse for the students who attend the preparation class optionally (although it 
is not compulsory). Furthermore, the participants claim that students lose motivation more and 
more as the classes get harder in the advanced stages. For instance, P2 says “Towards the 
middle of the semester, when they start to be unsuccessful in the exams, they start to cause 
problems when they start to think that they will already fail.” The participants also agree that it 
is often the external factors that result in low motivation among participants. As argued above, 
lack of a proper campus and frequent relocations among campuses decrease students’ 
motivation profoundly. 

At the end of the interviews, the participants were asked if there was anything else, 
they wanted to add. Two of them added further ideas. P3 shortly suggested that academic 
English should definitely be included. P1 on the other hand responded longer and made some 
suggestions. She argued that main course plus four-skills is too demanding. She explains it as 
follows: “When we have four skills, the main course and the English for academic purposes, 
this is too tiring for the students. The program should move with the main course book and 
only the needed skills should be added.” . She further suggests that “Too many hours were tiring 
for the students. More hours may not necessarily contribute to Ss’ learning.”. So, it may be a 
good idea to focus on quality rather than quantity. This is also true for the number of lessons as 
discussed above. There are around 22-28 hours of teaching to ensure the students learn English. 
However, this may counter react and overwhelm the students. 
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Discussion and Conclusion 

This study aims to evaluate the program implemented for the English preparatory 
classes at Istanbul Medeniyet University, School of Foreign Languages during the 2016-2017 
academic year. The results of the analysis conducted with the qualitative and quantitative data 
can be summarized in the light of the related literature as follows: 

The students were reasonably pleased with the English Preparatory Program 
implemented during the 2016-17 academic year. In the same vein, the instructors also think 
that the program is overall successful. This finding is consistent with the other studies 
exploring the curriculum evaluation of different language programs, which have revealed 
considerable satisfaction with the overall instruction (e.g., Unal et al., 2017; Tekin, 2015; 
Coskun, 2013). The study conducted by Demirtas and Sert (2010) with 1200 learners, for 
example, reports that most of the learners had no significant complaints about the program, 
and the English Language Preparatory Education met their needs. 

The students found Main Course classes quite efficient, whereas they were not satisfied 
with Pre-faculty, Terminology, Listening & Speaking, and Reading classes. According to the 
findings, the “Writing Course’ was the most effective and beneficial in the program. Tekin 
(2015) and Coskun (2013) focused on a desire for an improvement in listening and speaking 
courses, and Oztiirk’s study (2014) at a state university similarly emphasizes the “speaking” skill 
as the learners in the study indicated that “speaking skill is the least developed one during the 
program” (p. 122). Although the participants of our study found the Terminology course 
overwhelming, Unal et al. (2017) and Lee (2016) highlighted the importance of EAP and ESP 
courses based on the results they found. Considering the EAP needs of the first-year 
undergraduate students at a state university in Turkey, Kirkg6z (2006) developed a reading 
course based on the specialized corpus from academic texts that students are required to read at 
the first year of their studies. The interviews with the students revealed that a corpus-based 
reading course helped them to become familiar with essential lexical items of their field. 
Another very detailed study conducted by Kirkg6z (2009) shows that most students (93.5%) 
perceived a gap between the requirements of disciplinary courses and what they learned at the 
Foreign Language Center (p. 88). As a result, the findings of our study are not compatible with 
Kirkg6z’s study in this sense. Based on the results, it can be suggested that to explain the 
importance of learning the discipline-specific language and providing a suitable environment 
for the learners to experience EAP during the preparatory language program can help learners 
internalize it as a part of language learning. 

The students were more pleased with the packs and supplementary materials prepared 
by the Material and Curriculum Development Office compared with the commercial products. 
This comment is quite understandable since the related office had the opportunity to meet the 
needs of the local group directly, whereas the commercial products address a larger audience. 
Additionally, the instructors also accept that the commercial products could be overwhelming 
for low level learners although they are pleased with using them in their teaching. 

Although the students believed that the assignments were beneficial and appropriate 
for them in general, they found the online assignments insufficient due to the technical issues 
they had to cope with throughout the academic year. The instructors shared the same issue as 
well, and they suggest not to integrate online assignments to the program anymore. Unal et al. 
(2017) stated that more technology should be integrated into the curriculum, but it is essential 
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to remember that too many technical problems may frustrate learners and instructors, resulting 
in negative attitudes. 

The students considered the testing and evaluation processes appropriate for their level 
and coherent with the curriculum and also thought that the evaluation procedures were just. 
The Testing Office prepared all the tests within the school, considering the implemented 
program and the characteristics of the target group. Oz and Atay’s study with lecturers 
teaching at the preparatory program (2017) revealed that there is a mismatch between Turkish 
EFL instructors’ in-class assessment literacy and its reflection in practice. Therefore, the 
assessment process at Istanbul Medeniyet University, School of Foreign Languages, can be 
regarded as one of the strongest aspects of the implemented program. 

Although the students were content with both their effort and the instructors’ 
contribution to the program throughout the year, it is seen that they were more pleased with 
the instructors than their performance. The female students were significantly more content 
with the English preparatory program and the Writing Course than the males. Tekin (2015) 
and Coskun (2013) revealed that the participants were dissatisfied with the instructors’ 
teaching styles and demanded more communicative activities. However, this study revealed 
that the students were more critic of themselves than their teachers. 

The students who attended the program voluntarily were more content with it than 
the ones for whom it was a requirement to start their undergraduate program. However, they 
were less pleased with the Terminology course, which is quite understandable since the course 
was not offered to them and the medium of instruction would be Turkish in the following 
years, so they might have considered this course unnecessary. The students from B level classes 
were more content with the overall program as well as the following components than the ones 
from A level classes: Main Course, Reading and Terminology courses, Language Leader, 
Reading Explorer and supplementary materials, online assignments, use of English and 
listening tests, pair and group work activities, re-grouping students for every period. It is also 
understandable since A level students found it tougher to cope with the language learning 
process in many aspects due to their lower language competency. 

Harrison (1996) emphasizes the difficulty of curriculum renewal in terms of evaluation 
based on his external study in the country of Omani in which there was an extensive renewal 
including the development of books, materials, examination system, orientation programs for 
all teachers and even development programs for inspectors. This is also the case in Turkey and 
pre-service English teacher programs have changed considerably due to the changing position 
of English globally (Koksal & Ulum, 2018). Harrison (1996) suggested in the longitudinal data 
which were drawn upon four types of the collection that teachers and inspectors can 
contribute to material development and more importantly classroom interactions must be 
monitored not only by external visitors but by getting teachers and learners to reflect upon 
their teaching and learning, in a systematic and structured way. His suggestion of an approach 
for professional development involving both self and peer observation can also be a logical way 
for our recent study of program evaluation to provide a vehicle for the formative curriculum 
evaluation process (Harrison, 1996, p. 302). Since “Communicative Language Teaching” has 
been the widely used, sometimes, namely used, approach all over the world, the goals and aims 
of curriculum and language teaching programs have been shifted from more grammar-based 
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accuracy to fluency. Besides, the latest developments in “approaches and methods in foreign 
language teaching” umbrella term such as the post-method era, task-based teaching, and few 
others affect curriculum designers to cover different aspects of language. As discussed by 
Widdowson (1984) long ago, a learner’s task involves communicative competence in the 
language, whether the emphasis is on productive or receptive behavior. As a result, the aim of 
language learning should be to develop the underlying interpreting ability, and an integrated 
approach should be adopted to achieve this aim. Widdowson continues the discussion by 
stating that on the contrary to what this aim requires, language teaching courses commonly 
consist of units which are separate like language practice books and criticizes the language 
teaching materials. Students and instructors who participated in this study shared similar 
concerns with Widdowson (1984) in their own words. Thus, we can conclude that preparatory 
language programs and materials used in language preparatory programs must primarily focus 
on including language functions and how to help learners to transfer the functions accordingly 
in their studies. 

The following suggestions can be made to improve the English preparatory program 
following the findings summarized above: First of all, Terminology and Pre-faculty courses 
should be removed from the program since these courses are perceived as an extra burden, 
especially for the low level learners. Second, there is no need to have a separate Listening & 
Speaking course since Main Course is a skills-based class. The sections for these skills within 
the Main Course can be focused more efficiently. Third, Reading can be integrated with the 
Writing classes to reduce the boredom and to make it more meaningful for the learners. 
Fourth, students should be supported by supplementary materials that are designed considering 
their specific needs. Moreover, commercial coursebooks should be updated or changed to pick 
the more appropriate ones for the target group. Also, online assignments should be removed 
from the program since the technical issues do not seem likely to be fixed. In addition to these, 
students should be motivated to ensure successful learning as it is one of the most significant 
issues mentioned by the participant teachers. The interview data demonstrate that students lose 
motivation, especially towards the end of the program. Accordingly, this should be taken into 
consideration, and via further studies, the reasons for this problem can be unearthed which, 
then, may be used to increase their motivation. Finally, the program should be revised based on 
a needs analysis to set clear objectives rather than following a coursebook-based syllabus which 
is a similar problem revealed by some studies (e.g. Erdem & Atar, 2018) in the context of 
compulsory English courses at a higher education level. 
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